Department of Statistics University of Missouri - Columbia TR - MU - STAT - 2009 - 09 - 07

Support vector machine (SVM) has been successfully applied for cancer tumor classification based on thousands of gene expression measurements. A modification of SVM known as hybrid Huberized support vector machine (HHSVM) has been developed for the same purpose along with an in built gene selection mechanism with the help of elastic-net penalty. In this paper we develop a Bayesian formulation of the hybrid Huberized support vector machine for binary classification. For the coefficients of linear classification boundary we propose a new type of prior which can select variables and group them together simultaneously. Our proposed prior is a scale mixture of normal distributions and independent gamma priors on a transformation of its variance. We establish a direct connection between the Bayesian HHSVM model with our special prior and the standard HHSVM solution with elastic-net penalty. We propose a hierarchical Bayes and an empirical Bayes technique to select the penalty parameter. In the hierarchical Bayes model the penalty parameter is selected using a Beta prior. For the empirical Bayes model we estimate the penalty parameter by maximizing the marginal likelihood. A fast MCMC based algorithm to fit our model is also provided. The proposed model is applied on two simulated datasets and three real life gene expression microarray datasets. Results suggest that our Bayesian models are highly successful in selecting similarly behaved important genes in groups and predict the cancer class.